The present invention relates to computer-assisted animation in general and in particular to computer-assisted generation of inbetween frames that fill an animation sequence between key frames.
It is well known that an animated visual presentation can be created by creating individual static images (frames) and it is also well known that it requires a substantial amount of manual labor to create each frame in, for example, a feature length film. Often, what is done is that a subset of the frames at key action points are drawn first (the “key frames” in animation terminology) and then the remaining “inbetween frames” are created to fill in the intermediate motion between key frames.
“Inbetweening” is a process in the typical animation production pipeline, wherein an animator produces drawings at key frames that capture the heart of the animated motion and an “inbetweener” artist then draws the inbetween frames at specified time intervals between each pair of key frames that fill in the intermediate motion. In the general case, an inbetween frame is based on two or more key frames.
In general, each frame has an associated time and the animation is played by presenting frames in order according to their associated time. The time differences between adjacent frames need not be the same, but often it is, such as with feature films where each frame is designed to be presented 1/24th of a second after the presentation of the previous frame. Of course, frames are often not presented instantaneously and the same animation can run at different speeds, but for the purposes of generating animation, it is often sufficient just to refer to the “animation time” of a frame. For example, it is enough information to know just that one key frame is at time t=0, another key frame is at time t=1 and an inbetween frame is to be generated for time t=⅓ without considering the units of time.
Inbetween frames, or “inbetweens”, are considered “tight” when they are in the animation sequence between very similar key frames. Tight inbetweens are tedious for artists to draw because they require the greatest amount of precision and the least amount of artist interpretation. Where the animation comprises two-dimensional illustrations of three-dimensional scenes, moving objects in the scene can rotate or change shape and it is preferred that such movement be illustrated in the animation in a natural way (such as according to the laws of physics or according to the artificial or imagined “physics” of characters or objects in an animation). Herein, a tight inbetween is defined as one that is close in shape with little to no topological changes (e.g., as produced by visibility changes where a portion of the character is visible in one drawing and not in the other), but it should be understood that similar problems exist with non-tight inbetweens and might find benefit with similar solutions presented here.
The manual production of inbetween frames is both difficult and time-consuming, requiring many hours of intensive labor by skilled professionals. Furthermore, its cost is a significant part of the total production budget. A full-length feature animation might require inbetweening of over a million drawings, with typical ratios of 3-4 inbetweens for each key frame. Consequently, researchers have long sought an automatic solution. Despite a large body of work in this area spanning over four decades, no definitive solution exists. This is in part because the problem is ill-posed—there is no concise set of rules for producing high-quality inbetweens, especially in cases of complex motion and changing occlusion. The artist relies on his or her perspective of the 3D world in which the animated characters are embedded, as well as on an artistic vision for achieving the desired aesthetics. It would be insufficient to derive the three dimensional world from the drawings, even if it were possible, as most often the characters have their own, undefined, laws of physics and deformation.